Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a way to configure the RunJobFlowRequest #9

Merged
merged 1 commit into from
Feb 3, 2016

Conversation

EugenCepoi
Copy link
Contributor

This PR ads a back door for when spark-plug is not up to date with the features provided by the emr sdk.

@@ -19,7 +19,8 @@ class Emr private(credentials: Option[AWSCredentials]) {

def this(knownCredentials: AWSCredentials) = this(Some(knownCredentials))

def run(flow: JobFlow)(implicit config: ClusterConfig): String = {
def run(flow: JobFlow, configureRequest: RunJobFlowRequest => RunJobFlowRequest = identity)
(implicit config: ClusterConfig): String = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe provide a backwards compatible API here?

def run(flow :JobFlow)(implicit config: ClusterConfig): String = 
  run(flow, identity[RunJobFlowRequest])(config)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point but it is backwards compatible due to the default value (which basically does nothing)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh oops didn't see the default value, good call :)

ogrodnek added a commit that referenced this pull request Feb 3, 2016
Provide a way to configure the RunJobFlowRequest
@ogrodnek ogrodnek merged commit 546f3d1 into ogrodnek:master Feb 3, 2016
@ogrodnek
Copy link
Owner

ogrodnek commented Feb 3, 2016

Thanks for the commit!

@stephenh
Copy link
Collaborator

Okay, in theory a release with this change should show up in Maven central as com.bizo#spark-plug_2.10#1.2.6.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants